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ABSTRACT 

As change occurs in various health sciences programs, 
evaluational strategies should be developed so that adaptive 
decisions may be made. Evaluation models taking into account inputs, 
methodology, and outputs (Star\e, 1967, and Astin and Panes, 1971) 
need examination. Alternative measurement instruments for fornative 
and summative evaluations, including simulation techniques (McGuire 
and Babbott, 1967) , need consideration. It is particularly important 
to define adequate criterion levels of performance, including 
analysis of the implications of criterion*ref er enced measurement, 
(Popham and Husek, 1969), Systematic evaluation strategies should 
provide reliable and content- valid information from which decisions 
can be made concerning adequacy of student preparation as compared to 
content experts* a priori academic judgments. (Author) 
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With the existing demand for increased health care delivery, the various 
health science programs have attempted to expand and/or modify their curri- 
cular structure. This has been done primarily in response to a perceived 
need to provide alternative educational experiences which hopefully allow 
the student to proceed more rapidly through the curriculum whi"Je maintaining 
appropriate professional standards. 

These modifications have frequently been made in an attempt to obtain 
federal funding without the correspondinq systematic planning and/or evaluation 
needed for these programs. While models for specific implementation of stra- 
tegies do not exist, there are models which discuss in general terms the 
design and evaluation of educational programs Stake (1967), Metfessel and 
Michael (1967), and Astin and Panos (1971). Each of these models is merit- 
orious and the authors of this paper do not intend to present a new and unique 
approach to the problem. However, health science programs have some character- 
istics which need special attention and it seems appropriate to attempt to 
specify a systematic strategy which could be used to judge the proposed merits 
of an innovative curricular program. 

The authors will attempt to present the need for such a systematic 
strategy, the functions it may have and, finally, discuss some of its com- 
ponents. In this paper, which is basically developmental in nature, the 
authors will not attempt to be comprehensive but rather point out the areas 
they see as being of prime importance. 

NEED 

The need for a systematic evaluation strategy is mul ti -dimensional . 
Three major reasons for developing such a system are: 

1. Validation of funding (federal, state or private). 

2. Identification of student proficiencies and deficiencies. 
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and 

3. Accountability - which is broadly dei'ined here as the effective- 
ness of the varioir; educational processes beinq ijsed in a 
particular systeni. 
The large sums of money hein^j invested nece$,si tates that methods be 
developed to review the status of a given program in an objective way. A 
review will not always be oriented only at the determination of specific 
student performances but should include the degree to which the materials 
being developed are likely to facilitate the goals set forth in the planning 
document. Any modification to the original program should be defended with 
reference to specific constraints, not previously identified, which may 
necessitate alternate strategies. These might take the form of lack of 
physical resources, adminirtrative cooperation or other factors. A specific 
change should be logically defendable and evidence should be gathered regarding 
its congruity to the program as well as its effect on students' terminal per- 
formances . 

The second and most obvious need is that of identifying student perfor- 
mance capabilities. Without a systematic strategy specific student deficien- 
cies may not be identified, with the end result being the certification of 
an unqualified health professional. Equally important is the identification 
of student proficiencies which may lead to the conclusion that the program's 
emphasis is in the wrong area or areas. 

The emphasis here should be on the evaluational data being detailed 
enough to allow for diagnostic interpretation and adaptive curriculum modi- 
fication where indicated. A particular problem may be overlap or gaps in 
the content of the curriculum due to lack of coordination between the various 
teaching departments in a health science institution. 
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Finally, the eva ] uat i onal strategy should provide accountability data. 
At fir'jt impression this need appears to coincide directly with the first, 
validation of funding. However, accountability hero is used specifically 
to deal witfi the identification cf particularly efficient or inefficient 
techniques or pr^ocesses being used to transmit the content contained in a 
segn^cnt of the curriculum. Here again the data gathered should be detailed 
enough to allow for diagnostic decision making. The cost per student should 
be one factor carefully considered in this paradigm. 

FUNCTION 

The function of a systematic evaluation strategy is to provide multiple 
pieces of data which allow the educator to determine at various steps the 
status of students in the program with regards to the overall structure of 
the program. At each stoi^ in this process a feedback loop needs to be 
included which allows for adaptation of the program based on the available 
data. The three primary levels of assessment necessary to make compre- 
hensive decisions are presented diagrammatical ly in Diagram 1. 

An effective evaluation strategy must be a dynamic and continuous pro- 
cess. As is indicated in Diagram 1 the various levels of evaluation adapted 
from Bloom et al . (1971) are used to assess both student and program status 
from entry through final exit. 

The other important factors in this systematic approach are that the 
assessments are made at all levels and for all domains of the student's 
performance and that various checks are made on the system at each stage. 
The components of this system will be discussed in the next section. 

COMPONENTS 

Before identifying the various components which are needed in a functional 
system as outlined in Diagram 1, the relationship between curriculum design 

O 

ERsLC evaluational design should be clarified. Davis, et al . (1974) indicate 
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that qood ins tructio lal design occurs when goals are specified prior to the 
selection of methods, Prespeci f ication of goals allows the identification 
of objectives, instructional procedures and evaluation methods. Implementa- 
tion of a systGPiatic evaluational strategy provides feedback, regarding the 
objectives and instruction, that produces a cohesive and interactive curri- 
culum. A health science college, whether considering adoption of an inno- 
vative new program or continuance of a fairly traditional program, should 
look carefully at the total curriculur. 

What components should the evaluation segment of this overall design 
process have? Metfessel and Michael (1967) point out the necessity for 
m' tiple criterion-measures when evaluating a program. The function of 
multip= measures is to alleviate possible misinterpretation of data gathered 
from only one source which may have a substantial error component,. Since 
in a health science college both clinical ar.d didactic phases of the curri- 
culun: need to l)e evaluated, multiple measures must be identified for both of 
these phases . 

Diagram 2 presents in concise format some of the needed measures in each 
of the three ni. jor areas of evaluation. Each level of assessment in this 
model has some unique component thus reflecting the changing role that is 
played by the evaluation process at each level. 

Entry Level Assessments 

Entry level assessments are conducted across all domains of performance 
and are specifically referenced to the goals of the curriculum. Since appli- 
cants to health science institutions are generally highly qualified and since 
little behavior change can be expected prior to program entry, acceptance 
decisions are made on a normative basis taking into account both academic skills 
and affective predispositions. The addition of an affective dimension can 
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only be accomplished and justified when the goals of the specific institution 
are clearly explicated. 

Forma ti ve Assessments 

The formative evaluation process is the most complex, primarily because 
it is v/here the action is. Formative evaluation is in essence a continuous 
process which involves providing the student with multiple indications of his 
success with the subject iiiatter being studied. 

An emphasis is placed on criterion-referenced evaluation at this level 
for two reasons. The first involves the goals of health science education. 
A student who leaves the academic environment to enter one of the health care 
delivery fields is expected to not just be better than another graduate but 
rather to have "mastered" the appropriate skills which were established by 
content experts in the field. Criterion-referenced evaluation provides the 
framework to build this type of system. The second reason for using criterion- 
referenced evaluation is explicated by Pophan and Husek (1969). They argue 
that criterion-referenced tests are devised to make decisions both about 
individuals and treatments, e.g., instructional programs. If for instance 
a criterion-referenced measure based on a set of instructional objectives is 
developed and administered to appropriate learners after their completion of 
the instructional sequence dealing with the objectives, one could reach a 
decision regarding the effectiveness of the sequence. 

Simulation types of exercises could also be applicable to this type of 
system. Simulation of patient management situations as described by McGuire 
and Babbott (1967) provide the opportunity for a student to practice higher 
level problem solving skills. An additional benefit of simulation is that 
the learner can receive immediate feedback regarding the consequences of his 
selected approach, both positive and negative. 
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A last point of emphasis regarding the formative level is that affective 
measures should be carefully used to determine the student's growth in this 
area. The measures should be generally unobtrusive and involve informal 
observations as we^l as formal measures using carefully devised and standardized 
rating forms. These ratings should not be oriented towards making comparisons 
between students but rather at describing the student's behavior in an objective 
manner. 

Clinical performance should be judged using a carefully agreed upon set 
of criteria. Students should be encouraged to make self-evaluations and to 
compare these to the evaluations of the faculty. If the health science student 
is to progress to an independent stage by graduation, he must be given the 
opportunity and training to facilitate this independence. 

Summati ve Assessments 

The major concern of the authors is with evaluation which takes place 
as part of the learning process, however the final assessments of a student 
and how they ^elate to the ongoing evaluation should be briefly mentioned. 
In health science institutions these final assessments usually take the form 
of certification or licensure examinations. 

An appropriate technique to use in assessing non-clinical knowledge 
would be a norm-referenced standardized exam, but one which included a preset 
competency level. A student would be expected to score above the competency 
level to pass the exam but would be ranked against his peers as well. The 
ranking would serve the purpose of making judgments regarding the individual's 
relative standing at his exit from the program. It would be necessary to 
determine what basic goals the various health science institutions were 
trying to reach and to base this licensure examination on the comon goals 
if it was to be a valid measure. 
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Mock boards, within institutions, and board examinations conducted by 
an external licensing agency should both be based on clearly specified criteria. 
These criteria would be used in the formative stage to prepare the student 
for the end point examination. 

Program Eval uati on 

In conclusion, a few comments on program assessment seem appropriate. 
While all of the measures previously discussed should provide indirect or 
direct evidence O"^ program success there should be some additional 
comparisons made. 

Student ratings of alternative course niaterials and instructional tech- 
niques can be an invaluable data source. If the students are asked to make 
objective comparisons ar.d are provided with carefully designed rating scales 
to accomplish this procedure, the resultant information can be used to make 
appropriate modifications in course content and instructional technique. 

Besides student satisfaction comparisons should be made regarding effi- 
ciency, cost, flexibility and validity of alternative programs. Where possible, 
alternate instructional design should be tested in an experimental setting 
against the existing strategy. Where this is not possible the effectiveness 
should at least be determined by assessing student attainment of prespecified 
goal s . 
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